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ABSTPACT ^ . 

The JefferspB County Public School System in Colorado 
is conducting a study vhich^implements a digital time-^shared computer 
as support for a systems approach to instruction* This study 
currently involves one elementary school b^t it will support a total 
of thirteen schools in the future* The computer support includes 
comput^r*,?generat€d criteridfi tftst forms. The items for the test forms 
Are randomly .selected from a* test-item banJc* Additional computer 
support includes the use of a remote^ document reader for test scoring 
and updating student academic records* Th^ records are updated Using 
test results and teacher certification of mastery* Data from academic 
records is being studied to' determine its usefulness ^or 'formation of 
instructional groups in a: multi^unit elementary school* The design of 
feedback reports to teachers is being studied 'to optimi:!e the 
report's usefulness in instructional ^Sooesses* Thje effects on 
teachersV^^oles caused by the computer support of an existing 
instruct-ional system are being observed* (Author) ^ 
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COMPUTER SUPPORT FOR A SYSTEMS APPROACH TO INSTRUCTION; , 
PROBLEM STATEMENT AND DATA ENTRY TECHNIQUES ' 

' 'C . - < 

by Eugene ft. Collins and Dean C. Larsen 
Jefferson County Schools 
Lakewood-, Colorado 80?15 ' 

ABSTRACT 



The Jefferson County Public Sdiopls^is conducting a study 
which implcsnents a digital time-shared computer as support for 
a systems approach to instruction. This study presently involves 
one elementary school 'but it wil,l support a total of 13 schools in 
the future.^ The computer support includes computer generated 
crUerion ,ttest forftis. The items for the test forms are rapdqmly 
selected fr9m a test-item bank. Additional .computer support' 
includes the use of a remote document* reader for test'scoritig . 
and updating' student ai^ademic records. The records are updated 
using test results and teacher certification^of mastery! Data 
from academic records is being studied to determine its useful* 
ness for formation of instructional groups in a multi-unit 
elementary scKool, The design of f^dbacfc reports to teachers' 
'is being studied in order to optimize the report^s usefulness ' 
in instructional processes. The effects on teacher's roles 
caused by the comjjuter support of an existing instructional 
systeis are being observed. 
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COMPUTER SUPPORT FOR A SYSTEMS APP^ACH 70 INSTRUCTION 
.PROBLEM STATEMENT AND DATA EMTRY TECHNIQUES 
by Eugene A* Cfcllins and Dean C. Larseh 
' Many programs for indWidualizl^g education have been developed during 
the last ten years* An outcome of these experiences has been the identiflca- ' 
tlon or a problem which Is connnon to^all attempts to Inilivldualize* The 
problem Is one of data management* At any given time students in a particular 
group may be at many different places in the curriculum, each ^Student in the group 
haying different learning needs* Infon^ation about the learning needs of 
each Individual is usually available* HoWever, techniques have not been de- 
veloped for effectively managing this informa^non for optimizing the Instruc-^ 

tlon for each student/ * * - ' ' * 

' • ' ■ ' 1 y 

This paper is a description of a National Science Foundation project 

located In The Jefferson County Public Schools, Jefferson County, Colorado** 

The project Is designed to develop computer suppqrt for an existing Instruc- 

* ^ * . 

tlonal management information system in schools attempting to individualize 

. V ■ ■ 

instruction*^ The primary purpose^of the project Is to develop thfe.data manage- 
ment techniques, required *to group students with similar learning needs* , V 

' * * ^ * * \ 

The Jefferson County Public School district has defined curriculum In' 

language artsv mathen^atlcs, and reading by bating explicit student perform- 

ance objectives* Procedures for monitoring student* progress against these 

objectives liave been Identified* The programs have been Implemented through ^ 

grade a. In mathematics, criterion test Items have teen developed parallel 

to the objectives* 'Levels of .mastery have been established* Dependency ^ 

* ** " 
hierarchies between objectives have been identified, and multiple ^ learning 

activities have been designed which relate to these objectives* 

± 

1 * \ ' ' 

'The work reported in this paper was performed pursuant to Grant Number 
GY 11109 with- The National Science Foundation, Experimental Programs Group, 
Office of Experimental Projects and Programs* Any opinions, findings, con- 
clusions, or reconmiendations expressed herein are those of the authors and 
'do not necessarily reflect the views of N*S.F* 



Teachers are keeping records on .individual student perforaance against 

the objectives. However, these records on, individual ^tudent perrormance are 

not generally used as a basis for selecting appropriate learning Activities. 

Furthermore, in most schools, the formation Qf student groups for instruction 

is not accomplished through u$e of past performance records. That is to 

. $ 

say that student records are not widely used as input into the formation of 
groups of students for instruction or into the design of instructional stra- 
tegies. As a result, effective grouping of students for experience with 
appropriate learning Jctivities doesrnot always occur^ Students my/ie pro- 
vided instruction Jr))skins which they have previously mastered or for wMch 
they do not have the necessary prerequisite behaviors. 

Wfvile systematic diagnosis of sti/dent's individual needs may occur, 
this diagnostic information is not always available in a useable form when 
students arg grouped for instruction.. This is due primarily to the fact that 
instructional decfision makers have tooliuch data. Th^ do not have the pro- 
cedures nor the tipe necessary to process, 'it* , : 

The project described here is developing a solution to this problem. 
Fuqded by The National Science Foundation Office of Experimental Projects 
and Programs, Experimental Programs Group, the project h titled The Computer 
Support for a Systems Approach to Instruction . .A computer system which 
will allow instructional decision makers to process records on student per- 
formance ,is presently being developed. The outcome of the applicatior^^Jf 
this system will be the formation of studerjt groups for instruction. . 

The system is being developed'on a Hewletti-Pack'ard' 2000 series tima- 
shared computer. Conmunication and remote data entry are handled over tele- 
phone lines with a General Electric Tenninet 300 terminal and a Bell and Howell 
Jiark Document Reader. 
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Develoienient Is taking place at Normandy Elementary School. After 
refinenjent ' fjeld testing the system will be installed in a total of ^ 
eleven schools in the Bear Creek/Columbine Area of Jefferson bounty. These 
schools will be added in two v^aves over a period of four years after the 
.system implementation il completed in the development school. L 

TheiSystem i^s being developed. in ^a manner which will provide for the 
maintenance of performance information on c^iildren in any discipline. Currently* 
mathematics performance records of students in the development school are 
being managed on the computer. Fou^ years of performance hfstory are being 

1 - . L ^ 

maintained on each "Individual . In order to insure that these performance ^ 
records are current, two system components for updating them have been developed. 
The first component provides update by application of criterion referenced 
testing for mastery. The second compqnent allows update by teacher certtfica- 
tion. 

When teachers are Interested In testing students oyer any subset of 
objectives, in mathematics* they call for a test using the test generation 
subsystem. This subsystem generates ay recipe for the test by randomly selecting^ 
itdms for each objective from an item bank. The item bank contains approximately 
80% multiple choice *?tems. The recipe- produced by the computer lists the^ 
items to be used on the wst» the order InlWhich they are to appear»and the 
correct answer. Mw^ltiple choice-Items are lifted first followed by items 



-e nttecf first f( 
;. ThT\ recipe^ i: 



re'quiring more detailed student responsT^s. ThT\ recipe is used by an aide 
who constrifcts the test b^ drawing Items from the item bank. 

t . The decision has 'been made to use only two test items' per objective. 
Th^ Criteria for mastery is that both items must be answered correctVy. 



The l^est Is administered to students. They respond to multiple choice 

* « 

items on an answer car<I shown in figure 1. Students respond to non-multiple 
choice items on their test paper. These non-multiple chpice items are then 
graded and transcribed to the answer card by ah adult. An answer key is 
filled, out -from the test recipe. This is entered through the'mark docu- 
ment reader along with the student answer cards. Grading of the test ts 
done by. computer. The student'^ acaderatc performance records are automatically^ 
updated by the system, A sipmary of test results isT printed. out for teacher 
analysi^s. , a ^ 

The advantages of this procedure are numersus. First of all, therecis 
total flexibility regarding which objectives are tested together. Random 
selection of items from the bank is also assured. • Hore'than one 

fpnn of each test can be generated, Avi& there is no need fcrr computer storage 

■ 

of the answer .keys. Automated gradjng and record updaU4s provided along 
with enough human interface to allow for Ron-'multiple choice items. ; 

A second component of the system allows for bpdate of student records 

' ^ . J • 

through teachet^ observation anS subSequent^certificatioft Df mastery. This 

is a necessary cojiponen^ of the system because me^ry mathematics objectives 

^hrougl^ the second grade are not appropriately mejasured using gaper and 

pencil. StWent bi^havioV described by these objectives includes verbal response 

and application of manipulative devices. These behaviors must be observed 

and recorded by the teacher. Furthermore, this teacher observation and 

certification is ^in appropriate assessment of student behavior beyond 

second grade. . . ' ; 

; * ' • 

The need to update records by teacher observation and certification is 
being supported by the form shown in figure 2. ^The fcm is initially 
\MxiV. on the left side. The heading, containing studp' . na.-np, system number; 
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Name 



Date 

April 8, .1976 ' 



Level Objective 



D 
D 
D 
D 
I) 



E 
E 



8 
9 
16 
31 
33 



22 
42 



Mathematics 

Student 
Nuniber 
^59 



Mastery 



M 



# of Masters 
in Gi:dup 



43 
24 
29 
26 
21 



28 . 
12 \ 



w 
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figure 2 
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(nathematics level* etc. Is printed by the tennina!. Forms for each student 
in a* particular grmp are kept fogether-ln a foTder. Teachers observe*i^ai- 
vidua! student behavior. Mastery of particular objectives is then certified 
by marking the appropriate, code on the computer readable part of the form 

* ► / + 

on the fight side. Thes$ forms ^r;e read periodically by the mark document 

/ 

reader /Which has the capability of reading both and 8k" width forms. 
Astuclent records are up dated accordingly. 

As mentioned before, the form in figure 2 is initially blank on the 
left side. Alternative types of individual student 'profiles ^re bfeing 

* ' + d ^ - 

t ■* ^' 

printed remote^fy ty the terminal in this space. The project staff and 
teachers i^e workijig together ta determine which form^ of the indiviciual 
student profile are most helpful to^he teachers in. supporting their efforts 
in observing students. ' 

Now that many of thie mass data entry problems have been solved, tife pr1- 
mary task of the project is being pursued. The task is to iricrease the .use- 
ability of the data for teachers, , using, it to form instructional groups. 
The development school, as ifiell as many of the other project schools, is 
implementing Individually Guided Education^; Under this model, ^chools 

are organized on a'multiunit basis- Each.unit is composed of 1^0 to 150 stu- 
dents with up to three years age difference. The formation of such multiage 
units increases the heterogeneity of the unit with regard to sti^dent'.s previous 
background and experience. This increased variety enriches the learning • 
experience which can be brought to any instructional subgroup formed. within 
the unit.' This heterogeneity is important to learning in any discipline. ^ 
WitKin a muHiage unit, instructional subgroups are formed In the skill 
disciplines on the basis .0^ coiwnon, student l^^arning need. These skill 
disciplines are typically mathematics, reading, and language arts/ . Based on 
Hr observations and previous experience *with students, teachers identify^ 
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clusters of pbjective^ for which they think*'1nstruction is needed. Student's 
needs with regard to these*objectives are assessed.. If "instruction is needed, 
appropriate student groups are formed. 

The multiage unit organization along with the procedure for subgrppping 
on the basis of skill needs has the following advantages. Some of the 
heterogeneity preseat in the" totaK unit is maintained in the instructional 
subgroups. Furthermore, variability in student achievement in any subgroup 
is reduced. This/allows the opportuni^ty for planning an optical l,earning 
experience for each subgroup. 

student learniag needs. related to Any. cluster of objectives in mathematics ' 
are assessed in two ways; through diagnostic test procedures and through the ^ 
analysis of historical records of student performance. 

«# 

Vlli^n a team of teachers feels the need to regroup within their unit, 
the objectives for potential instruction, during th? next instructional period, 
are identified. A pretest is constructed over prerequisite objective^ using 
items from the test item bank. After administration of the pretest, the 
results-^are analyzed by.th^ teac^iers and the project slaffV Subsequently, 
student* subgroups are fwmed. The criteria for this subgroup formation 



are recorded as precisely as possible. ' ' 

While the teachers providing instruction, the project staff uses the 
computer to prt)be the historical performance data in an attempt to form 
similar groups. The stated grouping criteria are used as'-a guide for develo^ng 
alternative algorithms for subgrouping. These algorithms are progifainmed and 
■\ . executed. The composition of the suggested groups which result is then compared 

■ ■ ■ * ■ . r ; . .' 

with the composition of groups actual l\y" receiving instructibn. Pii the future 
\ this procedure will be repeated regularly, yhe desired outcome is the dis^ 
. covery of generalized algorithms for probing historical oe,*fon(),ifnce data to , 
produce instructional groups which are similar In compf/.rtiort %o uhose ob- ^ 
tained through pretesting. 

erJc . ^ . 10 * 
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Any successful appHcation of .computer technology to^ support instruc- 
tion;requi res attention to the interface problems which exist between the".* 
user,'System and the technical system- The l»vel of technical literacy on 
the^part'of educators is* of primary concern, they need enough technical 
literacV to understanjl both the power and limitations of the computer 
to comunicate precisely uith it* The required level of technical literacy 
may be different for each educator. This depends on the educator's role ■■ 
in the .schools and the person's relation to^ the information system. These 



various roles and relations must be observed. The necessary level of teCh- 
nical literacy must be identified and provided. 

The problem of understanding and precisely mod^ing thejjser$ original 
information system is of equal importance. Technical staff members must 
recognize their role as one of modeling the users infomation system rather 
than redesigning it to meet some external Qonstraints. Only if \ precise 
computer modelMs produced will meaningful computer service be provided. 

These interface problems are much more severe than generally recognized 
They ^re solved only through well planned procedures which increase person 
to person cotmiunication. , In ordef to increase person to person communi 
cation, the project has chosen the procedure of on-site development and 
slow expansion to other schools over ef'period of several years. 



